Click any tag below to further narrow down your results
Links
The first part of a series where the author writes data governance rules as code, builds a platform to test them, and has CI block any change that breaks a policy. It walks through how the data mesh principles map onto folders and checks.
- Governance rules live as files, and CI runs them on every pull request: a partner file that doesn't match its contract never reaches the warehouse, and a change that breaks another team's report fails and names that team.
- The project is organized around Zhamak Dehghani's four data mesh principles, mapped to folders (domains/, data_products/, platform/, governance/), with five domains and exactly one owner per dataset.
- The author favors end-to-end "proofs" over unit tests, since a proof checks that a policy still holds when all the pieces run together.
Data mesh hype cooled, but the underlying need for findable, trusted, owned data hasn't gone away. The author argues data products are shifting from a full enterprise restructuring into a practical discipline built into existing platforms, catalogs, and governance work.
- Data mesh asks for a big operating model change, and adoption stalled: a 2024 BARC study of 197 professionals found 20% had it underway and 24% planned it, down from 35% planning it in 2023.
- A data product is defined by a known consumer, a stated purpose, an accountable owner, and measurable use, not by the asset format itself.
- KPMG's 2025 survey of 250 executives found 92% saw data products as critical, yet only 35% reported extensive value, a gap between intent and execution.
- AI agents raise the stakes: without clear ownership, meaning, quality, and permitted use, they make weak decisions faster, so governed, portable product definitions matter more.